63 research outputs found
Is BERT Really Robust? A Strong Baseline for Natural Language Attack on Text Classification and Entailment
Machine learning algorithms are often vulnerable to adversarial examples that
have imperceptible alterations from the original counterparts but can fool the
state-of-the-art models. It is helpful to evaluate or even improve the
robustness of these models by exposing the maliciously crafted adversarial
examples. In this paper, we present TextFooler, a simple but strong baseline to
generate natural adversarial text. By applying it to two fundamental natural
language tasks, text classification and textual entailment, we successfully
attacked three target models, including the powerful pre-trained BERT, and the
widely used convolutional and recurrent neural networks. We demonstrate the
advantages of this framework in three ways: (1) effective---it outperforms
state-of-the-art attacks in terms of success rate and perturbation rate, (2)
utility-preserving---it preserves semantic content and grammaticality, and
remains correctly classified by humans, and (3) efficient---it generates
adversarial text with computational complexity linear to the text length. *The
code, pre-trained target models, and test examples are available at
https://github.com/jind11/TextFooler.Comment: AAAI 2020 (Oral
Deep Learning for Text Style Transfer: A Survey
Text style transfer is an important task in natural language generation,
which aims to control certain attributes in the generated text, such as
politeness, emotion, humor, and many others. It has a long history in the field
of natural language processing, and recently has re-gained significant
attention thanks to the promising performance brought by deep neural models. In
this paper, we present a systematic survey of the research on neural text style
transfer, spanning over 100 representative articles since the first neural text
style transfer work in 2017. We discuss the task formulation, existing datasets
and subtasks, evaluation, as well as the rich methodologies in the presence of
parallel and non-parallel data. We also provide discussions on a variety of
important topics regarding the future development of this task. Our curated
paper list is at https://github.com/zhijing-jin/Text_Style_Transfer_SurveyComment: Computational Linguistics Journal 202
Mo\^usai: Text-to-Music Generation with Long-Context Latent Diffusion
Recent years have seen the rapid development of large generative models for
text; however, much less research has explored the connection between text and
another "language" of communication -- music. Music, much like text, can convey
emotions, stories, and ideas, and has its own unique structure and syntax. In
our work, we bridge text and music via a text-to-music generation model that is
highly efficient, expressive, and can handle long-term structure. Specifically,
we develop Mo\^usai, a cascading two-stage latent diffusion model that can
generate multiple minutes of high-quality stereo music at 48kHz from textual
descriptions. Moreover, our model features high efficiency, which enables
real-time inference on a single consumer GPU with a reasonable speed. Through
experiments and property analyses, we show our model's competence over a
variety of criteria compared with existing music generation models. Lastly, to
promote the open-source culture, we provide a collection of open-source
libraries with the hope of facilitating future work in the field. We
open-source the following: Codes:
https://github.com/archinetai/audio-diffusion-pytorch; music samples for this
paper: http://bit.ly/44ozWDH; all music samples for all models:
https://bit.ly/audio-diffusion
How Good Is NLP? A Sober Look at NLP Tasks through the Lens of Social Impact
Recent years have seen many breakthroughs in natural language processing
(NLP), transitioning it from a mostly theoretical field to one with many
real-world applications. Noting the rising number of applications of other
machine learning and AI techniques with pervasive societal impact, we
anticipate the rising importance of developing NLP technologies for social
good. Inspired by theories in moral philosophy and global priorities research,
we aim to promote a guideline for social good in the context of NLP. We lay the
foundations via the moral philosophy definition of social good, propose a
framework to evaluate the direct and indirect real-world impact of NLP tasks,
and adopt the methodology of global priorities research to identify priority
causes for NLP research. Finally, we use our theoretical framework to provide
some practical guidelines for future NLP research for social good. Our data and
code are available at http://github.com/zhijing-jin/nlp4sg_acl2021. In
addition, we curate a list of papers and resources on NLP for social good at
https://github.com/zhijing-jin/NLP4SocialGood_Papers.Comment: Findings of ACL 2021; also accepted at the NLP for Positive Impact
workshop@ACL 202
When does aggregating multiple skills with multi-task learning work? A case study in financial NLP
Multi-task learning (MTL) aims at achieving a better model by leveraging data and knowledge from multiple tasks. However, MTL does not always work – sometimes negative transfer occurs between tasks, especially when aggregating loosely related skills, leaving it an open question when MTL works. Previous studies show that MTL performance can be improved by algorithmic tricks. However, what tasks and skills should be included is less well explored. In this work, we conduct a case study in Financial NLP where multiple datasets exist for skills relevant to the domain, such as numeric reasoning and sentiment analysis. Due to the task difficulty and data scarcity in the Financial NLP domain, we explore when aggregating such diverse skills from multiple datasets with MTL can work. Our findings suggest that the key to MTL success lies in skill diversity, relatedness between tasks, and choice of aggregation size and shared capacity. Specifically, MTL works well when tasks are diverse but related, and when the size of the task aggregation and the shared capacity of the model are balanced to avoid overwhelming certain tasks
Navigating the Ocean of Biases: Political Bias Attribution in Language Models via Causal Structures
The rapid advancement of Large Language Models (LLMs) has sparked intense
debate regarding their ability to perceive and interpret complex
socio-political landscapes. In this study, we undertake an exploration of
decision-making processes and inherent biases within LLMs, exemplified by
ChatGPT, specifically contextualizing our analysis within political debates. We
aim not to critique or validate LLMs' values, but rather to discern how they
interpret and adjudicate "good arguments." By applying Activity Dependency
Networks (ADNs), we extract the LLMs' implicit criteria for such assessments
and illustrate how normative values influence these perceptions. We discuss the
consequences of our findings for human-AI alignment and bias mitigation. Our
code and data at https://github.com/david-jenny/LLM-Political-Study
- …